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ABSTRACT 


* 


This  final  report  describes  the  research  carried  out  by  members  of 
the  Decision  and  Control  Sciences  Group  at  the  Electronic  Systems  Labora- 
tory, M.I.T.  during  the  time  period  February  1,  1972  to  January  31,  1977 
with  support  extended  by  the  Air  Force  Office  of  Scientific  Research  under 
Grant  AF-AFOSR  72-2273. 

The  principal  investigators  were  Professor  Michael  Athans  and 
Professor  Sanjoy  Hitter.  The  contract  monitors  were  Lt.  Colonel  W.J.  Rabe 
and  Captain  C.L.  Nefzger  of  the  AFOSR  Directorate  of  Mathematical  and  Infor- 
mation Sciences. 

Research  was  carried  out  on  the  following  main  topics: 

1.  Modelling  and  Identification  of  Stochastic  Systems 

2.  Structure  of  Mon-linear  Stochastic  Systems 

3.  Linear  and  Non-linear  Filtering 

4.  Failure  Detection  Methods 

5.  Stochastic  Control 

6.  Adaptive  Control 

7.  Control  and  Estimation  Problems  for  Systems  with  Time  Delay 
Technical  details  of  the  research  may  be  found  in  the  reports,  theses,  and 
papers  cited  in  the  references.  A list  of  publications  supported  wholly  or 
partially  by  this  grant  is  included  at  the  end  of  this  report. 
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0.  Introduction 

In  this  report  we  describe  the  work  on  stochastic  and  adaptive  systems 
which  has  been  done  during  the  period  February  1,  1972  to  January  31,  1977 
supported  by  the  grant  AF-AFOSR  72-2273.  The  report  is  divided  into  two 
parts.  In  Part  1 of  this  report  we  describe  the  work  that  has  been  done 
during  the  period  January  1,  1976  to  January  31,  1977.  A number  of  new  inves- 
tigations has  been  initiated  during  this  period.  In  Part  2 of  this  report  we 
give  a summary  of  research  that  has  been  accomplished  during  the  period 
February  1,  1972  to  January  1,  1976.  Details  of  this  can  be  found  in  the  list 
of  publications  which  has  been  transmitted  to  Captain  Nefzger  earlier. 

Research  has  been  conducted  on  the  following  main  topics: 

Modelling  and  Identification  of  Stochastic  Systems 

Structure  of  Non-linear  Stochastic  Systems 

Linear  and  Non-linear  Filtering 

Failure  Detection  Methods 

Stochastic  Control 

Adaptive  Control 

Control  and  Estimation  Problems  for  Systems  with  Time  Delay 
We  remark  that  this  work  has  applications  to  a wide  class  of  aerospace  problems. 
0.1.  Contributions  Made  in  this  Research  Program 

We  feel  that  the  major  contributions  of  this  work  have  been  the  following: 

i)  Modelling  of  y noise  and  filtering  in  the  presence  of  y noise 

ii)  Performance  Bounds  for  Non-linear  Filters  using  Information  Theory  ideas 
and  development  of  an  informational  framework  for  the  study  of  non-linear 
filtering 

iii)  Explicit  construction  of  finite  dimensional  optimal  non-linear  filters 
for  a class  of  non-linear  systems  which  are  important  in  aerospace  appli- 
cations 
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iv)  Important  insights  into  Consistent  Estimation  and  Dynamic  System 
Identification 

v)  Stability  Criteria  for  Feedback  Systems  with  coloured  noise  gains 

vi)  Filtering  theory  for  systems  with  time  delay,  both  when  the  delay 
is  known  and  when  it  is  unknown  and  has  to  be  estimated 

vii)  Solution  of  the  one-step  delay  Linear  Quadratic  Gaussian  problem 

viii)  Development  of  a theory  and  algorithm  for  estimation  and  control  of 
finite  state  probabilistic  systems  over  an  infinite  horizon  using 
finite  memory  estimators  and  controllers 

ix)  Insights  into  steady-state  optimum  stochastic  control  of  linear  systems 
with  random  parameters 

x)  Development  of  a theqrv  of  estimation  and  control  for  a large  class  of 
infinite-dimensional  linear  systems 


Description  of  Research  done  during  the  Period 
January  1,  1976  to  January  31,  1977. 


1.  Introduction 


Research  has  been  conducted  on  the  following  topics: 

i)  Modelling  and  Identification  of  Stochastic  Systems 

ii)  Nonlinear  Estimation 

iii)  Stochastic  Stability 

iv)  Geometric  Theory  of  Control  of  Linear  Stochastic  Systems 

v)  Robust  Linear  Quadratic  Design 

vi)  Linear  Quadratic  Gaussian  Stochastic  Control  Problem 

vii)  Adaptive  Control 

viii)  Control  of  Finite-State  Stochastic  Systems 

ix)  Finite-State  Compensation. 

This  work  is  summarized  in  this  section  of  the  retjort.  Further  details  can 
be  found  in  the  publications  and  theses  referred  to  in  the  text. 


A 
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2.  Description  of  Research 

2.1.  Modelling  an  .if ication 

In  order  to  do  filtering  and  control  using  state-space  methods  it  is 
necessary  to  have  a Markov  model  of  the  stochastic  system.  Even  if  a Markov 
model  were  available  the  parameters  of  the  system  may  be  unknown  and  may  have 
to  be  identified  using  noisy  measurements.  We  have  began  investigating  these 
fundamental  aspects  of  modelling  of  stochastic  systems. 

2.1.1.  Stochastic  Realization  Theory 

For  finite-dimensional  constant  linear  systems  a sophisticated  reali- 
zation theory  is  now  available.  For  stochastic  linear  systems  very  little 
seems  to  be  available.  In  joint  work  with  G.  Picci  [1]  a theory  of  stochastic 
realizations  has  been  initiated.  The  goal  of  this  research  is  to  construct 
a theory  analogous  to  the  theory  of  Kalman  [2]  for  deterministic  linear  systems 
which  would  enable  us  to  give  an  intrinsic  construction  of  the  innovations  repre- 
sentation of  second  order  stochastic  processes.  This  work  is  being  done  by 
Professor  Mitter.  Further  clarification  of  the  results  obtained  in  [1]  have 
been  made  and  will  be  reported  in  a forthcoming  publication. 

2.1.2.  Information,  Consistent  Estimation,  and  Dynamic  System  I lent i fi cat  ion 
Mr.  Yoram  Baram  and  Professor  N.K.  Sandell,  Jr.  have  recently  completed 

a study  of  the  asymptotic  behavior  of  various  system  identification  methods. 

The  study  was  motivated  by  another  research  project  in  F.SL  in  which  the 
dynamic  equations  of  an  aircraft  were  identified  during  flight  for  purposes 
of  adaptive  control.  Due  to  computer  limitations,  the  approach  was  to  identify 
an  appropriate  model  from  a small  finite  set  of  models  corresponding  to  various 
flight  conditions  of  the  aircraft.  For  this  application,  it  was  particularly 
obvious  that  the  true  system  (even  neglecting  nonlinearities)  would  almost 
never  bo  a w 'tuber  ot  tf.e  model  f>ot . Thus  t he  qu  >..<  . •>«  of  tin  behavior  ot 
vim  'u  id  •iticat  : : hniquos  when  the  ti  . 1 no!  an  admissibli 
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estirute  arises  naturally  for  this  application.  A rel  :ted  question  is  how 
to  choose  a finite  number  of  models  to  best  represent  the  true  system.  Note 
that  both  of  these  questions  are  quite  distinct  from  the  usual  question  of 
consistency  of  identification  methods;  the  problem  of  consistency  explicitly 
assumes  that  the  true  system  is  an  admissible  estimate. 

With  this  background  as  motivation,  the  asymptotic  behavior  of  parameter 
estimates  and  the  identification  and  modeling  of  dynamical  systems  were  investi- 
gated. Measures  of  the  relevant  information  in  a given  sequence  of  observa- 
tions were  defined  and  shown  to  possess  useful  properties,  such  as  the  metric 
property  on  the  parameter  set.  The  convergence  of  maximum  likelihood  and  re- 
lated Bayesian  estimates  for  general  observation  sequences  were  investigated. 

The  situation  where  the  true  parameter  is  not  a member  of  a given  parameter 
set  was  considered  as  well  as  the  situation  where  the  parameter  set  includes 
the  true  model.  The  finite  parameter  set  case  was  emphasized  for  simplicity 
in  the  convergence  analysis,  but  the  results  were  extended  in  general  terms  to 
the  infinite  parameter  case.  It  was  shown  that  under  uniqueness  conditions 
on  the  output,  statistics  of  linear  dynamical  systems  identification  procedures 
converge  to  the  true  model  if  it  is  a member  of  the  given  model  set.  If  the 
true  model  is  not  a member  of  the  set,  then  the  estimates  converge  to  a model  in 
the  set,  closest  to  the  actual  system  in  the  information  metric  sense.  Station- 
ary and  non-stationary  systems  were  considered.  Rates  of  convergence  in  the 
mean  were  obtained,  and  the  separate  contributions  of  the  stochastic  and  the 
deterministic  parts  of  the  input  to  the  convergence  rates  were  shown.  The 
analysis  also  suggested  methods  for  approximating  a high  order  system  by  a low- 
order  model  and  for  selecting  a representative  model  from  a given  model  set, 
applicable  to  infinite  and  even  non-compact  model  sets. 

These  results  have  been  reported  in  the  t’h.D.  thesis  of  Mr.  Baram  [1] . 
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R.E.  Kalman,  P.L.  Falb  and  M.A.  Arbib,  Topics  in  Mathematical  System  Theory 
McGraw  Hill,  New  York,  1963. 


Y.  Baram,  "Information,  Consistent  Estimation  and  Dynamic  System  Identifi 
cation",  Massachusetts  Institute  of  Technology,  Cambridge,  Massachusetts, 
Ph.D.  Thesis,  ESL-R-718,  November  1976. 


Non-linear  Estimation 


fficient  Optimal  Estimators  Using  Spherical 


'ouraer 


During  the  past  year  v;e  have  continued  our  efforts  to  develop  efficient 


and  accurate  estimation  schemes  for  nonlin 


work  are  described  in  detail  in  [1 


] and  further  successful  experimental 


Specifically  work  has  been  concentrated 


in  two  directions.  In  the  first  of  these  [1,3]  we  have  applied  technique 


from  the  theory  of  spherical  harmonics  in  order  to  obtain  simple  nonlinear 


filter  design  methodologies  for  certain  systems.  Recently,  this  methodology 


was  applied  to  the  design  of  a system  for  tracking  the  motion  of  a point 


iphere  giver,  noisy  measurements  of  position.  In  all 


cases  tested  our  filter  outperformed  or  at  least  matched  the  performance 


of  the  extended  Kalman  filter  and  a linear  minimum  variance  filter.  In 


addition,  our  filter  is  far  simpler  to  implement.  Given  these  promising 


these  ideas  to  a detailed  look  at 


wo  are  con 


satellite  tracking  and  inertial  navigation  problems 


The  second  direction,  as  discussed  in 


of  estimation  for  certain  finite  state  systems.  As  many  nonlinear  filtering 


systems  are  to  be  implemented  in  a discrete  fashion — i . e.  on  a digital  com- 
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wiLh  extremely  large  numbers  of  states,  and  the  combinatorics  _,f  the  opt  iinal 
estimation  problem  are  imposing.  It  is  precisely  this  issue  that  we  address. 
Specifically,  with  the  aid  of  the  tools  of  finite  Fourier  analysis,  we  have 
developed  extremely  efficient  algorithms  for  estimation  in  certain  special 
cases.  Future  work  will  involve  loosening  the  restrictions  under  which  the  e 
techniques  can  be  applied. 

Estimation  for  Systems  with  Unknown  Time  Delays 

The  problem  considered  in  [4]  is  one  of  practical  importance  in  a 
variety  of  applications.  A transmitter  sends  out  a signal  which  is  sensed 
by  one  or  more  sensors,  but  the  transmission  tine  from  the  transmitter  to 
each  sensor  is  only  described  in  a probabilistic  fashion.  In  addition, 
the  signal  may  be  reflected,  leading  to  multipath  observations.  We  would 
like  to  estimate  both  the  transmitted  signal  and  the  transmission  time 
delays.  In  [4]  we  have  considered  several  simplified  versions  of  this 
problem,  have  obtained  implementable  optimal  estimation  equations  in  cer- 
tain cases,  and  have  uncovered  some  of  the  key  issues  involved  in  the  design 
of  recursive  systems  for  such  problems.  Work  will  continue  in  the  develop- 
ment of  usable  algorithms  for  problems  of  this  type. 

References 
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and  Control , Clearwater  Beach,  Florida,  December  1976. 


-10- 


1 


2.2.2.  Stable  Non-linear  Observers 

Mr.  Safonov  and  Professor  Athans  have  completed  work  which  discusses 
a new  type  of  nonlinear  estimator  design,  designated  the  Suboptimal  Nonlinear 
Observer  (SNO) . The  SNO  constitutes  a simple,  practical  alternative  to  the 
Extended  Kalman  Filter  for  nonlinear  estimation. 

A hybrid  between  the  Kalman  Filter  and  the  Nonlinear  Observer*,  the  SNO 
consists  of  a nonlinear  internal  reference  model  of  the  system  dynamics  with 
observations  entering  via  a memoryless  gain  acting  on  the  residual  error  be- 
tween predicted  and  actual  observations.  The  distinguishing  feature  of  the 
SNO  is  the  way  in  which  its  residual  gain  is  chosen:  it  is  chosen  to  be  the 

Kalman  filter  gain  for  a nominal  time-invariant  linearization  of  the  system. 
This  choice  leads  to  an  estimator  design  which  maintains  to  a first  approxi- 
mation the  minimum- variance  optimality  of  the  Extended  Kalman  Filter  but  is 
much  simpler  in  structure. 

The  results  obtained  concerning  SNO  estimators  include  analytically 
verifiable  conditions  for  a SNO  design  to  be  assured  of  being  nondivergent. 
These  conditions  are  constructive  in  the  sense  that  they  provide  the  engineer 
with  information  that  is  helpful  in  making  design  modifications  necessary  to 
correct  divergence  problems. 

The  SNO  estimator  design  procedure  has  been  found  to  be  robust  in  the 
sense  that  the  class  of  nonlinear  systems  for  which  SNO  designs  are  assured  of 
being  nondivergent  is  large;  this  tolerance  of  nonlinearity  is  mathematically 


*T.-J.  Tarn  and  V.  Rasis,  "Observers  for  Nonlinear  Stochastic  Systems", 
Trans,  on  Automatic  Control,  Vol.  AC-21,  August  1976,  pp.  441-448. 
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dual  to  the  previously  reported  robustness  of  linear-quadratic  state-feedback 
systems  against  variations  in  open-loop  dynamics  find  unmodelled  nonlinearity. 

It  has  been  proved  that  substitution  of  the  estimates  generated  by  a 
noadivergent  SN'O  for  the  true  values  in  an  otherwise  stable  feedback  control 
system  can  never  destabilize  the  closed-loop  system.  This  result  has  important 
practical  implications  regarding  the  utility  of  SNO  estimators  for  state  re- 
construction in  nonlinear  optimal  and  suboptimal  feedback  control  systems. 

2.3.  Stochastic  Stability 

Following  the  work  of  Willems  and  Blankenship  [1]  on  the  stochastic 
stability  of  linear  systems  with  a white-noise  multiplicative  feedback 
gain,  Martin  [2]  studied  the  more  realistic  case  of  a scalar  colored-noise 
multiplicative  feedback  gain.  This  problem  is  closely-related  to  the  problem 
of  assessing  closed-loop  performance  due  to  a stochastic  control  law,  where 
the  control  law  itself  may  be  random  if  it  depends  on  past  observations  of 
system  outputs  or  on  parameter  estimates.  The  case  of  scalar  colored-noise 
gains  studied  by  Martin  (see  also  Martin  and  Johnson  [3,4])  requires  sub- 
stantially more  than  a mere  formal  extension  of  the  earlier  white-noise 
results.  The  algebraic  complexity  of  the  moment  expressions  and  their  bounds 
previously  prevented  extension  of  Martin's  results  to  the  multivariable  case; 
during  1976  this  hurdle  w as  overcome  and  results  will  be  presented  in  a future 

1 

paper  in  the  IEEE  Transactions  on  Automatic  Control,  supported  by  this  grant. 

) 

One  of  the  most  remarkable  features  of  such  systems,  implicit  in  early 
work  of  Brockett  and  willems  [5],  is  that  moment  stability  does  not  imply 
alrnost-sure  stability;  in  fact,  the  conditions  for  stability  of  the  higher- 
order  moments  become  progressively  more  severe  (see  [3], [4]).  Related  problems 
have  been  studied  by  Marcus  [6].  There  are  reasonable  parameter  values  for 
which  almost  every  sample  path  i ; uniformly  bounded  (it;  the  initial  state) 
bn  all  mo;  nv;  -come  urihou.  !.••!.  T;,<  signifi  • «•:  th  • •«  <*ui  i i ly  did  n » 

L — , J 
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becom-2  apparent  until  the  recent  investigations  of  Ku  and  Athans  [7],  reported 
elsewhere  herein.  For  good  reasons,  the  use  of  mean-square  estimation  and 
control  criteria  has  been  common  in  both  linear  and  nonlinear  stochastic  control 
problems.  Approximate  solutions  of  such  problems  have  often  presumed  moment 
stability.  Particularly  in  cases  where  uncertain  or  slowly-varying  parameters 
enter  multiplicatively , mean-square  criteria  may  yield  closed- loop  stochastic 
stability  requirements  which  are  too  conservative.  These  considerations  may 
require  a fundamental  reassessment  of  our  formulation  of  stochastic  control 
problems;  continuing  research  is  under  way. 
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2.4.  Geometric  Theory  of  Control  of  Linear  Stochastic  Systems 

Professor  Mitter  has  begun  development  of  qualitative  theory  of  control 
of  linear  stochastic  systems  analogous  to  the  geometric  theory  of  control  of 
linear  deterministic  systems  as  developed  by  Wonharn  [1] . The  first  task  in 
such  a development  is  the  need  to  define  the  concept  of  a stochastically  un- 
observable subspace.  This  in  turn  necessitates  a detailed  study  of  the 
Riccati  equation  without  any  assumptions  of  controllability  and  observability. 

An  examination  of  the  literature  or.  the  Riccati  equation  has  shown  that  very 
little  is  available  in  this  direction  and  what  is  available  seems  to  be  in- 
correct. No  results  have  been  obtained  so  far  but  an  understanding  of  the  nature 
of  results  to  be  obtained  has  been  gained. 

Reference 

1.  W.M.  Wonharn,  Linear  Multivariable  Control,  Springer  Lecture  Notes,  1975. 

2.5.  Closed-Lo.  p Stability  Robustness  of  Nui t i loop  I, .'-Pytimil  Design 
New  results  have  been  obtained  characterizing  the  robust!  ■ ss  of  the 

closed- loop  system  stability  property  of  multiloop  LQ-design  is  the  presence 
of  perturbations  in  the  feedback  gains  as  well  as  in  the  open-loop  system 
parameters.  These  new  results  unify  and  generalize  known  stability  margin 
properties  of  LO-design.  The  new  robustness  measures  are  expressed  ex- 
plicitly in  terms  of  the  weighting  matrices  in  the  quadratic  cost  index. 

These  explicit  relationships  provide  a new  basis  for  studying  the  effects 
of  the  choice  of  cost  criteria  on  the  closed-loop  stability  robustness  of 
the  corresponding  "optir.al"-de3ign.  In  particular,  the  questions  of  system 
reliability  (feedback  integrity)  to  failures  in  the  feedback  loops  can  be 
investigated  using  the  new  results  we  obtained. 

The  results  of  our  research  are  reported  in: 
f’.K.  Wong,  K.  Athens,  "Glosud-Loop  .'-stability  Robustness  of  Multi  loop 
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2.6.  Linear-Quadratic  Gaussian  Stochastic  Control  Problems 

2.6.1.  There  are  countless  formulations  of  LQG  stochastic  control  problems, 
and  a great  number  of  these  have  been  analyzed  in  periodicals;  yet,  surprises 
of  fundamental  import  may  be  found  here.  In  a series  of  papers  we  have  studied 
control  law  simplifications  which  result  when  the  stochastic  control  problem 
is  nearly  singular,  in  the  sense  that  plant  process  or  observation  noises 
may  be  negligably  small.  In  these  cases,  the  standard  Kalman  filter/quadratic 
regulator  solution  will  be  unsatisfactory  in  steady  state;  when  process  noise 
is  absent,  the  filter  gains  go  to  zero  (the  filter  is  oblivious  to  new  infor- 
mation) , and  in  the  absence  of  observation  noise,  they  become  infinite  (because 
they  depend  on  the  inverse  of  the  covariance) . In  Platznan  and  Johnson 
it  has  been  shown  that  an  output- feedthrough  term  can  be  determined  in  such 
a way  that  the  foregoing  problems  are  avoided;  this  requires  an  innovations- 
decoraposition  of  the  control  law  and  an  extension  of  the  LQG  performance  index 
to  penalize  the  innovations  part  of  the  control.  The  extended  problem  does 
not  give  rise  to  the  anomalies  of  the  usual  LQG  problem,  which  are  often  an 
enigma  in  practical  computation  of  optimal  regulators;  it  may  provide  a general 
setting  which  ensures  uniform  convergence  of  stochastic  controls  in  the  case  of 
small  plant  or  observation  noise.  In  the  case  where  some  observations  are  noise- 
less, Rom  and  Sarachik  demonstrated  that  the  Bryson-Johanson  filter  followed 
by  the  Kalman  gains  is  an  optimum  solution  of  the  singular  LOG  problem,  i.o.  a 
rodu  ed-order  filter  (equivalent  to  a minimal-order  observer)  can  be  used.  In 
[2],  we  have  studied  the  algebraic  structure  of  this  problem  and  showed  that, 
in  principle,  it  is  possible  to  determine  the  optimum  compensator's  transfer 
function  directly  and  uniquely  from  the  plant  transfer  function  and  the  noise 
and  performance  index  parameters.  In  (3],  wo  have  shown  that  the  solution  of 
a dual  problem,  where  the  process  noise  is  singular  is  a dual-observer  based 


compensator  whoso  optimum  gains  are  in  complete  algebraic  duality  to  the 
optimum  gains  of  Korn  and  Sarachik.  These  results  provide  insight  for  the 
design  of  low-order  approximate  stochastic  control  in  the  case  where  some  of 
the  plant  and/or  observation  disturbances  are  very  small. 

References 

1.  L.K.  Platzman  and  T.L.  Johnson,  "A  Linear-Quadratic-Gaussian  Control 
Problem  with  Innovations-Feedthrough  Solution",  IEEE  Trans.  Auto. 

Control , Vol.  21,  No.  5. 

2.  P.  Blanvillain  and  T.L.  Johnson,  "Invariants  of  Optimal  Minimal-Order 
Observer-Based  Compensators",  Proc.  1976  IEEE  Conference  on  Decision 
and  Control , pp.  186-190,  (Clearwater , Florida) , Reference  No.  10. 

3.  P.J.  Blanvillain  and  T.L.  Johnson,  "Specific-Optimal  Control  with  a 
Dual  Minimal-Order  Observer-Based  Compensator" , submitted  to  Int.  J. 

Control , August,  1976. 

4.  T.L.  Johnson,  "Finite-State  Compensators  for  Physical  Systems",  ESL 
Technical  Memorandum  No.  653,  April,  1976. 

2.6.2.  Optimal  Estimation  and  Stochastic  Control  for  Interconnected  Systems 

Two  of  the  major  problems  with  the  standard  Kalman  filter  and  linear- 

quadratic-Gaussian  regulator  are  that  it  is  quite  difficult  to  solve  the 

required  Riccati  equations  in  order  to  determine  the  optimal  gains  and 

also  the  required  on-line  computation  for  these  designs  may  be  exorbitant. 

Recently  [Ref.  2,  Section  2.2.1.)  we  have  begun  to  investigate  a class  of 

stochastic  control  and  estimation  problems  for  which  efficient  solutions  can 

be  obtained  with  the  aid  of  fast  Fourier  transform  techniques.  These  systems 

consist  of  a collection  of  identical  subsystems  which  are  coupled  toqether  in 

a symmetric  fashion.  Our  initial  results  for  these  systems  are  quite  promising, 

and  we  plan  to  continue  the  study  of  such  systems  and  the  applicability  of  our 

techniques  to  systems  which  may  not  exactly  satisfy  the  structural  symmetry 

assumptions  imposed  in  the  reference  cited  above. 
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2.7.  Adaptive  Control  of  Systems  with  Purely  Random  Parameters 

Professor  Athans  and  Mr.  Richard  Ku  have  continued  their  work  on 

Adaptive  Control.  Progress  has  been  made  on  the  problem  of  optimal  control 

of  stochastic  linear  systems  with  stocluistic  parameters.  In  particular,  the 

system  is  assumed  to  be  a linear  plant  with  randomly  varying  matrices  A and  B 

perturbed  by  additive  white  Gaussian  noise, 

Xf+1  = Atxf  + Bfcut  + _t  t 0,1,..., T-l 

It  is  assumed  that  the  randomly  varying  parameters  and  plant  noise 

are  independent  of  each  other  for  all  t.  Perfect  measurements 

y = x are  assumed. 

— t — t 

The  controls  are  restricted  to  the  class,  of  non  randomized  control 
strategies  of  the  form 

— t = 

l(t-l 


and  U = {'Jo'*  • • 


whore  V 

Given  the  system  structure  and  information  pattern  above,  it  is  desired 


to  find  the  optimal  control  law  { u^  : t = 0,1,..., T-l}  such  that  the  expected 

value  of  the  quadratic  cost  functional 

T-l 

J = x'F  xm  + I x'px  +u'R  u 
— T -T  — s s-s  -s  s — s 

s=0 


is  minimized.  V7e  assume  that  0 >0  and  R >0. 

s — s 

It  is  found  that  for  the  one-dimensional  problem,  that  the  optimal 
control  law  is  of  the  form 

Ut  = “Gt  -X-t 

where  the  non- random  control  gain  is  given  by 
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The  scalars  K arc  computed  from  trie  backward  difference  equation 

„ K2  AT.  (t)  + a b ) 2 

, . -2,  t+1  ab  t t 

K = + K (E  (t)  + a ) - -r 

t+1  aa  R +K  (E  (t)  + b2) 

t t+1  bo  t 


K (T)  = F. 

It  is  of  interest  to  compute  the  optimal  stationary  control  for  a linear 
time-invariant  system  with  stationary  statistics.  For  the  scalar  system,  it 

is  found  that  the  solution  to  the  Riccati  equation  may  not  be  finite. 

( E + ab) " 
r-  -2  ab 


If  we  define  m = E + a 


-2  ' 
E,,  + b 
bb 


For  m > 1,  the  backwards  solution  K becomes  unbounded.  The  expected  optimal 
cost  is  therefore  unbounded.  Hence,  the  optimal  control  system  is  asymptotically 
unstable  in  the  mean  square  sense.  Since  has  a constant  value  even  for 
large  Kt,  the  sample  path  xt  may  be  stable,  but  the  mean-square  sense  stability 
of  x cannot  be  established. 

/ 

Necessary  and  sufficient  conditions  for  a randomly  varying  system  to 
be  mean-square  asymptotically  stable  is  that  the  combination  of  levels  of 
parameter  uncertainties  giver,  by  m be  less  than  or.  3.  The  theorem  holds  for  the 
more  complex  multivariable  system  as  well. 

From  this  important  result,  various  concepts  of  stochastic  stability, 
stochastic  controllability  and  observability  may  be  established.  Furthermore, 
the  control  law  for  the  system  with  independent  parameter  variations  clarifies 
the  notion  of  dual  control  and  the  inherent  caution  and  probing  phenomenon 
in  the  interaction  of  control  and  identification. 


2.8.  Control  of  Finite  State  Stochastic  Systems 


2.8.1.  Finite  Memory  Control  and  Estimation  of  Finite  Probabilistic  Systems 
A finite  probabilistic  system  (FPS)  is  a stationary  discrete-time 
controlled  stochastic  dynamical  process,  having  finite  input,  output,  and 
(internal)  state  sets.  The  partially-observable  Harkov  decision  process  is 
an  example  of  such  a system.  FPS  formulations  provide  a convenient  framework 
for  the  study  of  problems  of  state  estimation,  statistical  decision,  or  control, 
where  state  information  is  available  only  through  a finite  memory less  channel, 
and  observation  dynamics  may  depend  on  the  inputs  selected. 

Notions  of  reachability  and  detectability  in  FPS's  (similar  to  controll- 
ability and  observability  in  linear  systems)  have  been  made  precise.  It  has 
been  shown  that  every  FPS  can  be  reduced  to  components  that  are  either  reach- 
able and  detectable,  or  transient,  or  null-recurrent. 

It  has  been  well  known  that  the  information  vector  (whose  i-th  entry  is 
the  a_  posteriori  probability  that  the  system  is  in  state  i)  is  a sufficient 
statistic  (for  the  estimation  of  future  dynamics  given  past  inputs  and  Outputs) . 

A contraction  property  of  the  information  vector  transition  function  has  been 
exploited  to  obtain  procedures  for  e-optimal  (arbitrarily  close)  approximation 
of  the  information  vector  by  a deterministic  time-invariant  finite-memory  observer. 
Each  observer  state  corresponded  to  a particular  configuration  of  most  recent 
input-output  pairs.  The  average  error  achieved  by  such  an  approximation  has 

-T 

been  bounded  by  the  expression  (m/m^)  , where  and  I are  parameters  associated 

with  the  observed  system,  and  m is  the  number  of  observer  states. 

Control  problems,  in  which  the  average  reward  has  been  maximized  over  a 
discounted  or  undiscounted  infinite  horizon,  may  be  solved  by  an  iterative  pro- 
cedure which  has  been  niven  the  name  perceptive  dynamic  programming.  Successively 
weaker  assumptions  that  the  controller  "perceives"  unavailable  state  values  has 
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transformed  the  problen  into  a sequence  of  formulations  which  nu\y  be  solved  by 
dynamic  programming.  Each  solution  obtained  in  this  manner  has  been  used  to 
construct  a feasible  controller  formulation,  taking  the  form  of  a deterministic 
time-invariant  finite-state  autonat-  >n.  Monotone  geometrically  convergent 
bounds,  containing  both  the  supremum  fen  inle  performance  and  that  of  the  current 
design,  have  also  been  obtained.  Computation  may  be  terminated  when  these  bounds 
have  become  sufficiently  close,  or  when  the  number  of  controller  states  has 
become  excessively  large.  Although  computing  a solution  by  perceptive  dynamic 
programming  may  require  considerable  tine  and  storage,  both  have  been  roughly 
proportional  to  the  number  of  controller  states  allowed  in  the  final  iteration; 
thus  the  cost  of  controller  design  has  reflected  the  cost  of  controller  imple- 
mentation. 

This  procedure  was  applied  to  idealized  problems  of  machine  maintenance 
and  computer  communication,  both  of  which  had  been  investigated  by  other  re- 
searchers. The  first  problem  was  solved  exactly;  a design  suitable  cost  to 
the  optimum  was  obtained  for  the  second  problem. 

Details  of  this  work  can  be  bound  in  the  doctoral  dissertation  of 
L.K.  Platzman  entitled  "Finite  Memory  Estimation  and  Control  of  Finite 
Probabilistic  Systems"  written  under  the  direction  of  Professor  Hitter. 

2.9. 2 . Finite-State  Compensation 

Aside  from  1 i near-quadratic— Gauss'  t.  stochastic  control  problems,  the 
only  substantial  class  of  processes  for  which  stochastic  controls  might  be 
considered  "computable"  are  Markov  chains.  In  this  case  policy  improvement 
algorithms  may  converge  in  a finite  number  of  steps  (the  number  of  possible 
control  laws  is  denumerable,  under  appropriate  assumptions).  Furthermore, 
such  control  laws  are  directly  implementable  by  finite-state  sequential 
machines  having  sufficient  memory.  Problems  for  which  time  and/or  states 
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programming  problems  for  the  numerical  approximation  of  the  feedback  operator, 
which  is  generally  infinite-dimensional.  The  ideas  presented  in  [cf.  Ref.  4, 
Section  2.0.1.]  relate  to  this  problem.  We  have  proposed  that  in  cases  where 
the  physical  variables  of  a continuous  system  may  be  apriori  bounded,  then  it 
is  in  principle  possible  to  uni formly  approximate  the  optimum  control  law  by 
a finite-state  (possibly  asynchronous)  machine.  A decomposition  of  the  feed- 
back mapping  is  also  proposed.  While  the  ideas  put  forward  in  [Ref.  5,  Section 
2.6.1.]  are  based  on  extensive  practical  experience,  they  must  yet  be  made  more 
precise,  initially  through  consideration  of  some  realistic  examples. 


Summary  Description  of  Research  Conducted  During  the  Period  1972-1976 


A study  was  undertaken  to  model  gyroscopic  noise  which  had  been  experi 


mentally  found  to  have  a spectral  density  exhibiting 


behavior.  The  final 


objective  was  to  do  filtering  in  the  presence  of  gyroscopic  noise 


A linear  infinite  dimensional  model  of  gyroscopic  noise  was  constructed 


The  filtering  problem  in  the  presence  of  gyroscopic  noise  was  solved  com- 


pletely. It  was  shown  that  under  appropriate  detectability  and  stabiliza- 


bility  hypotheses  the  filter  was  asymptotically  stable.  The  filter  is  natu 


problem  of  finite  dimensional  approximation 


of  the  filter  was  investigated  and  upper  and  lowe 


rformance  bounds  of  the 


L.B.  Horowitz,  "Optimal  Filterii 
June  1974,  Massachusetts  Institu 
setts. 


■g  of  Gyroscopic  Noise",  Ph.D.  Thesis 
.e  of  Technology,  Cambridge,  Kassachu- 


Clearwater,  Florida,  December  1976 


1.2  Non-linear  Filterin 


Research  on  various  aspects  of  non-linear  filtering  was  done  by  Profess 


K.P.  Dunn,  J.  Galdos  and  S.  Marcus 


1 Performance  Rounds  for  Non-linear  Filtering  using  Information  Theory 


The  importance  of  the  nonlineu 


iltcring  problem  in  aerospace  applica 


wile  re  one 


(acceptable  distortion)  can  be  achi 
information  quantities.  The  bridge 
is  therefore  the  relation  between  uncertainty  as  characterized  by  information 
quantities  and  uncertainty  as  characterized  by  mean  square  (MS)  distortion 
in  the  filtering  context. 

The  fundamental  work  in  the  information  theory  context  relating  informa- 
tion to  distortion  w as  done  by  Shannon  [3]  who  formulated  what  is  now  known 
as  Rate  Distortion  Theory.  Work  aimed  at  relating  information  theory  to 
dynamical  systems  and  filtering  has  been  done  by  Gray  [6] , Toms  [7] , Goeblick 
[4] , and  Keidemann  [1] . The  common  difficulty  found  in  these  studies  is  that 
they  attempt  to  inbed  dynamical  systems  and  filtering  into  the  classical  in- 
formation theory  context  which  is  asymptotic  in  nature  and  with  interval 
performance  indices.  In  contrast  modern  filtering  is  essentially  a real  time 
(causal)  problem  and  the  filtering  performance  index  is  a point  performance 
index  (MSE) . In  this  study  the  filtering  problem  has  been  embedded  into  a 
modified  information  context  which  is  far  more  natural  and  which  allows  the 
clearer  establishment  of  the  relation  between  information  and  MS  distortion. 

Potentially  this  formulation  may  allov;  the  imposition  of  an  information 
framework  for  the  design  and  comparison  of  nonlinear  filters. 

At  a more  immediately  applicable  level  this  study  has  resulted  in  the 
development  of  a lower  bound  on  filtering  MSE  based  on  information  quantities 
(rate  distortion  theory).  A number  of  such  lower  bounds  have  appeared  in  the 
literature  but  all  of  them  suffer  from  the  drawback  that  they  lose  their 
usefulness  when  considerable  nonlinearity  exists.  Recently  Zakai  and  Ziv  [8] 
pro. ’.need  a bound  bused  on  rate  distortion  theory  which  in  addition  to  the 


linearity  consideration  was  not  applicable  to  discrete  time  systems  and  thus 
not  applicable,  for  example,  to  radar  tracking.  The  bound  described  in  this 
study  is  based  on  the  Bucy  representation  [5]  which  in  addition  to  being 
easily  applicable  to  discrete  systems,  provides  a tighter  bound  on  the  rate 
distortion  function  and  hence  on  MSE. 

In  terms  of  usefulness,  a lower  bound  on  MSE  is  an  excellent  design  tool 
both  before  and  during  the  filter  design  — allowing  the  engineer  to  assess 
whether  the  design  is  feasible  for  the  particular  sensor  under  consideration 
as  well  as  providing  an  absolute  with  which  to  compare  the  different  possible 
designs.  Details  of  our  work  can  be  found  in  [9] . 
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it  is  we  tl-known  that  in  nonlinear  estii  pi  blerns  tht  determination 

of  the  tine  optimum  estimator  or  its  pcrfcuYP.nce  require-,  a ce:..;  ututionally- 
ir.tractuble  infinite- dimensional  on-line  calculation.  This  has  i-o tivated 
work  on  the  deter  uiv.iion  of  sub-optimal  estimators  and  the  study  of  their 
per  form  .nice.  Put  the  exact  evaluation  of  th.  performance  of  a sub-optimal 
estimator  is  in  general  not  an  easy  problem.  Recently,  Rhodes  and  Oilman 
! 1 ] , [2]  have  established  upper  and  lowei  perfon  nee  bounds  on  filtering 
problems  associated  with  a special  class  of  nonlinear  systems,  which  are 
computational ly-trae!  able,  rigorously-derived  and  osymptot ical ly- ti ght . The 
main  objective  of  the  current  research  is  to  extend  these  results  to  smoothing 
and  prediction  problems  associated  with  the  same  class  of  nonlinear  systems, 
and  also  to  include  the  situation  when  system  and  measurement  noises  are 
correlated. 

The  particular  systems  studied  are.  systems  with  cone-bounded  nonlincar- 
ities  in  the  sense  that,  when  modeled  by  T tc7  differential  equations,  they 
contain  drift  coefficients  which  are,  to  within  a uniformly  J.ipschitz  reside..!, 
jointly  linear  in  the  system  state  and  externally  applied  control.  The  esti- 
mation bounds  obtained  in  this  research  are  uppei  bounds  on  the  performance  <■) 
some  simple,  almost  linear  predictor  and  lower  bounds  on  the  estii  it  ion  error 
covariance  attainable  by  any  smoother  or  any  predictor,  including  th  optimt  ... 
They  are  solely  determined  by  the  cone-bounded  nature  of  th  nonl incari ties 
and  can  be  easily  computed.  Furthermore,  they  are  ind  pen  ent  ol  th<  conti  3 
or  control  law.  AI  L hound  ; have  the  proj  ’t  y tliat  th-  v oonver*  uni  lorrdy 
to  the  optimum  perform  nice  of  the  nominal  linear  i rol  1..,-.  .v  th  ■ nonlinanrities 


t end  to 


■ - 
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1.2.3.  Analysis  and  Estiisal  Lon  of  lien  li  n-.-.o  B t<  - 1 \ •tic  -i  :.;•••  Posse:-  ; • 
Certai  n Tvi  j;i  of  Structures 

Professor  Will. sky  with  the  aid  of  Dr.  Marcus  and  other  graduate  students 
has  done  work  in  the  area  of  stochastic  analysis  for  not  J i near  systems 
possessing  ccuain  types  of  structure.  The  major  results  obtained  in  this 
area  arose  out  of  the  consideration  of  the  fo]  loving  bi.liiv.  nr  estimation 


problem 

N 

X ( t)  - [A  3 £ A.f..  (t)  ]X(L)  (1) 

i=l  3 3 

£(t)  - F£(t)  + w(t)  (2) 

where  wo  obsc rve 

z^t)  = Ilg(t)  + v (t)  (3) 

z2(t)  - llx(t)  -I-  v (t)  (O 


The  cast  in  which  wo.  observe ■ the  process  sc  is  motivated  by  sti  .pdown 
navigation  problems,  in  which  f,  can  be  interpreted  as  an  angular  velocity 
vector.  This  clam  of  problem!  1,  * ) ' < l d<  d oi ■■  oi  the  first  large  class  ol 
nonlincai  fill.'  ring  p» . bloi  . f«>»  vl.R  h th<  > . < rist  <•'  lief,  fd:  ' ; 


optimal  solutions . 


The  basic  method  behind  tilt  work  report  ed  in  [30-12]  is 
the  expansion  of  the  solution  of  (1)  in  a Volte- -a  series 


x(t) 


go(t) 


(s)f  . (.'■■)  dx 
r 


N ft  Ci  . . 

+ }2  I I |i~J(T,s)C.  («)C^  (f)dsdt.  -t  ... 

i,j=-i  JO  JO  1 3 


(f>) 


One  of  the  major  results  obtained  is  a set  of  f ini tc— dimensional  nonlinear 
stochastic  equations  for  the  optimal  estimate  of  each  tern  in  thin  series . 
Thus  if  the  series  is  finite  — which  .is  equivalent  to  an  algebraic  condi- 
tion on  the  jV  in  (1)  — one  obtains  a finite-dimensional  optimal  estimate 
for  x.  Generalizations  of  this  to  some  cases  in  which  (i>)  nerd  not.  be  finite 


are  given  in  [1-3].  In  addition,  in  these  reforenec-s  it  is  shown  that  i nv 
nonlinear  systems  have  representations  as  Voltena  series  as  in  (5),  and 
optimal  estimation  schemes  for  these  are  derived. 

In  the  cases  in  which  the  series  (5)  does  not  lend  to  finite— dimensional 
optimal  estimators,  one  can  use  an  approximation  technique  to  truncate  the 
series.  The  utility  of  this  — especially  for  the  strapdown  problem  ---  is 
being  investigated. 

If  one  wants  to  consider  estir.i  ion  given  the  i '-ensure':. ont  (-3),  the  probli 
is  much  more  difficult.  These  models  arise  in  synchronous  communication, 
satellite  tracking,  and  incorporation  of  measurements  t.uch  re  from  a.  star 
tracker  into  an  estimate  of  attitude.  The  problem  (1),  (2),  (1)  lend:,  immedi- 

ately to  infinite-dimensional  filter  equations.  Tor  the  phase  tracking  emb- 
lem, Willsky  had  earlier  used  a Fouriei  series  method  to  obtain  sab. ipt.imal 
i si  in  . to i. : ; . Woi  t has  i.i  dotii  to  obt  .li  pi  ioi  i j i . ; boa  Is  ! •,>  t !>• 


in  addition, 


ph.ine  tracking  systems  and  to  develop  alternative  do  .•'.ion:'.. 

Marcus  1 1 ] has  extended  those  concent.:  to  the  u.  e ion  oi‘  ha:  i »nic-anc ly!..!  o- 
bnsod  nonlinear  filters  tor  sate.llitf  tracking  and  attitude  estimation. 

Reference?; 

1.  S.I.  Marcus. , "Estimation  and  Analysis  of  Nonlinear  Stochastic  Systems", 
Ph.D.  Dissertation,  also  Report  EJL-R-601 , June  1975. 

2.  S.I.  Marcus  and  A.S.  Willsky,  "Alyekraic  Si  ructuro  and  finite  Di:  . n?,ion:.l 
Nonlinear  Estimation"*  Proc.  Symposium  Algebi  lie  .•  yst« Theory,  Udine, 
Italy,  June  1975;  extended  version  submitted:  to  S TAM  • : . CcuT_tro_I. 

3.  A.S.  Will  sky  and  S.I.  Marcus,  "Estimation  for  Certain  Nonlinear  Stochastic 
Systems,"  Seminai  ?;  T HI  ~ , Rocquencourt , Erance,  May  1975. 

1.3  Filtering  for  Delay  Differential  Syst  ms 

Work  on  filtering  for  linear  and  nonlinear  time  delay  systems  has  been 
done.  This  work  lias  been  done  by  Dr.  R.W.  Kwony,  Prof.  A.S.  Wil  1 r.ky , and 
Prof.  S.K.  Hitter.  Briefly,  the  problem  undei  consideration  is  described 
by  the  functional  differential  equation 

dx (t)  - f(x  )dt  + dw  ( t ) (1) 

where  x^_  is  the  state  of  the  system  defined  by 

x (0)  - x(t  + 0)  -T  < T < 0 (2) 

Thus,  (.1)  includes  some  time  delay  effects..  Similarly,  the  measurement 
equation  is 

dn(t)  -•  h(x  )dt  + dv(t)  . (3) 

and  in  this  model  one  can  consiikr  "multij  ith"  or  "echo"  effects  — c.o. 
h(x  ) = ax (t)  + b>: (t  - i)  (d) 


The  b jor  results  obi  ined  ar<  s olloi  . . i . . ■ i . i ini  1 ■ . ] repi 


conditional  estimate 


sentation  formula  has  been  obtained  for  tin  • 

E N’  (xt ) | : (s)  , 0 < s < t]  (5) 

of  any  function  <£>(>:  ) of  the  trajectory  piece  x . Recursive  filtering  equa- 
tions are  shown  to  exist  in  the  linear  case  and  in  the  nonlinear  case  when 
h and  f satisfy  certain  smoothness  conditions  ( e . g . , pure  point  del  ays  cannot 
be  accomodated  and  one  needs  some  type  of  "blurring") 

r o 


h(xt)  - 


p(0)x(t  + 6)d9. 


In  the  linear-point  delay  case,  however,  a full  set  of  filti  r equations 
are  rigorously  derived,  and,  in  addition,  the  stability  of  the  steady-state 
filter  is  shown  when  the  only  delays  art  in  the  dynai.d cs  (1)  and  when  certain 
observability --type  conditions  arc  satisfied. 


These  results  lu.  vo  be<_n  extended  to  the  extremely  important  p:  obi  cm  when 
the  time  delay  T is  not  known . Examples  of  this  arise  in  various  signal 
processing  problems  in  which  one  is  trying  to  estimate  the  distance  from 
transmitter  to  receiver  (or  reflector)  given  that  multipath  effects  may 
corrupt  the  received  signal. 

Using  methods  to  study  the  infinite-time  quadratic  cosl  problem  for 
infinite  dimensional  systems  Professor  bitter  (in  collaboration  with  Dr. 

K.B.  Vinter,  formerly  at  M.T.T.  and  now  at  Imperial  College)  has  been  able 
to  prove  the  stability  of  the  filter  (-in  the  linear  case)  using  hypotheses 
of  stabiJ inability  and  detectability.  The  stabi Usability  and  detectability 
conditions  can  be  verified  using  matrix  tests.  The.  filter  obtained  is  hover-  r 
infinite  dimensional  and  work  is  now  in  prognss  to  understand  lien,-  the 
filter  can  be  implemented  using  a ires,  multiplier  and  do1  ay  ehr  i t:.. 


-31- 


Ri ■ f t.']  m . . ■ ; 


1 . Kw  : t ■ u:tv  ! Pi  ..  >rtios  ’ !'•  • ' • 1' j on  of  D 1 terns , ) • i ' . l . 

Thesi s , Repot t ,-l<-G14,  El ectronic  Syste  : 1 . sacbusetts 
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2.  S.K.  Mitfcer  and  R.B.  Vinter,  "Pi.lt  ring  for  ? . ; , Stochastic  Hereditary 

Differential  Systems,  in  Control  Theory,  Numerical  Mi  thods  and  Computer 
Systems  Modelling",  edited  by  A.  Banuoucsan  an  1 J.L.  Lions,  Springe r-Verlae 
New  York , pp.  1-2],  1975  (also,  pit  united  at  lb  ■ Internationa 1 Symposium  on 
Control  Theory , Numeric  t Metho  . a d c< mpute > ysi  n Mod  , Institut  de 
Recherche  D’ Informati  jue  it  D* Automatique,  Prance,  June  1974.) 

1.4  The  Et. . b.i  1 i tv  of  I.i  near  r .rtc  wits  Nun-V.'hit  e Hgl  lit)!  i < ..live  Kon-i; 


Often,  the  mathematical  model  of  a dynamical  system  require:-  the  consid- 
eration of  uncertain  elements.  Historically,  emphasis  has  been  placed  on 
studying  systems  in  which  disturbances  enter  acid.it  i vely , but  many  control 
problems  include  disturbances  that  can  bo  more  naturally  modeled  as  uncertain 
multiplicative  gains.  If  the  multiplicative  ip. in  is  of  the  white  noise  type, 
necessary  and  sufficient  conditions  for  the  stability  of  the  system  have  bc-en 
derived.  These  results  have  been  extended  where  the  multiplicative  gain  is 
of  correlated  noise  typo. 

This  research  has  been  done  by  Dr.  David  Martin  in  conjunction  with 
Professors  Johnson  and  Willsky . Systems  with  multiplicative  correlated  noise 
in  the  gain  elements  of  the  feedback  path  include  oscillators , certain  phase- 
locked  loop  configurations , and  rccuisivi  computational  algorithms , as  well  as 
human  opei ator-controllcd  systems.  Hence,  stability  criteria  for  such  system; 
may  be  of  considerable  interest  in  Air  Tore.  Applications.  The  use  of  colored- 
noise  gain  elements  .is  both  more  realistic  arid  technically  much  more  difficult 
than  the  previous  iy- treated  case  of  whi  t e-noise  gains..  Two  theoretically 
results  of  fundamental  importance  have  boon  establi  shed,  in  the  eonl.i  nuour.- 
t.ime  case,  a rigorous  proof  of  existence  of  solutions  to  the  stocnar.tic 
differential  equal  ions  chnvaei  mining  ?.n«  h sy  hems  h<  b*  en  c \ lop«-'.  Li  1 h- 


discrete- t.i in*:  c,.r>e,  tlx:  Markov  property  of  the  solution  s.<  quence  has  In  un 
proved.  both  of  these  theorems  establish  the  class  of  functions;  to  which 
solutions  of  the  equations  must  belong,  which  is  a necessary  prerequisite* 
for  proper  stability  results. 

For  the  special  case  of  continuous- time  first-order  system.;  with  first- 
order  colored  noise  gain  processes,  explicit  stability  criteria  are  available. 
'Die  effect  of  correlation  Lime  and  average  gain  can  be  evaluated  both  for 
"almost  sure"  and  moment  stability.  Certain  values  of  gain  ar.d  correlation 
time  can  be  found  which  yield  "stable"  sample  paths  but  for  which  the  system 
is  unstable  .in  the  mean-square  sense.  An  approximate  analog  simulation  of 
this  situation  was  carried  out  in  order  to  gain  a qualitative  understanding 
of  the  behavior  of  such  sample  paths . To  the  extent  that  effects  such  as 
nonideal  noise  generation,  amplifier  saturation,  and  multiplier  imperfections 
can  be  ignored,  it  appears  that  the  sample  trajectories  in  this  case  o::’  :hlt 
"burst"  disturbances,  i.e. , periods  of  guiescncc  interspersed  with  episode:; 
of  large  drift  or  oscillatory  disturbances.  In  this  qualitative  sen:. a , the 
simulation  is  in  agreement  with  published  studies  of  long-term  stability 
of  commercial  electronic  oscillators. 

For  first-order  continuous-time  systems  with  higher-order  noisy  tain 
processes  bounds  on  stability  regions  in  parameter  space  could  bo  d .'lived. 

Such  bounds  are  unique  in  their  depcuder.ee  on  the  bandwidth  of  the  gain 
process . 

in  discrete  time,  moment  stability  for  syst  ns  with  finite  correlation 
times  in  the  gain  process  has  been  studied,  and  stability  legions  have  been 
i den4  Lfiod  by  directly  calculating  g owih  »vf<  ;;  of  tin-  moment  equation!  as 

do 


a f irn*  l ion  c ( 1 ,1,  nt  nur . 


l and  1 > i - 1 li  of  con  . In’.iot,  t ir  . 


Pm  i • 1: 
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1.5.  Fail"":'  p.-tc-Uion  Me t.hods 

Professors  A.S.  Willsky  and  N.R.  Fandell,  Jr.  and  Pr.  K.P.  Dunn  have 
initiated  studies  into  the  development  of  on-line  failure  detection  and  comp- 
ensation and  false  moaauromont  re  a ction  methods.  Tlio  original  direction  oi 


this  fork  inis  coa 


,;,,c  from  NASA/L.mgloy , and  we  are  aiming  our  work  at  the  design 


of  a system  that  can  lc  used  in  digital  fly-by-wire  control  systems. 


The  cipptOcH'h  that,  ha*;  boon  chosen  is  a 


i ■ ralized  LiX<  Lihood  r<  ti 


technique  [1]  m which  we  make  all  detection  decisions  based  on  the  residual 
of  a single  Kalman  filter. 

Computer  algorithms  tor  the  generation  of  the  failure  detection  system 
gains  for  a wide  variety  of  actuator  and  sensor  failure  modes  have  I von  de- 
veloped. In  addition  methods  for  reducing  the  amount  of  on-line  compute  ?. 
storage  and  computation  required  by  the  CLK  system  have  been  investigated 
and  we  have  examined  the.  effects  of  parameter  variations  on  system  performance 

Reference 


1.  A.S.  Willsky  and  II. L.  Jones,  "A  Generalized  Likelihood  Ratio  Approach  to 
State  Estimation  in  Linear  Systems  Subject  to  Abrupt  (ranges  , 

1974  XEEE  Confes  r on  Decisit  a and  Coni  col ; submitted  to  1RSI rans.  on 

Aut.  Cen troi • 

1.6.  St. < >ch. is  t ic  Cent  i ;ol 

The  following  problems  in  stochastic  control  have  been  invesi i ■ / ,u.ed : 

i)  Control  of  finite  state  memory  stochastic-  systems 

ii)  One-step  delay  IQC  problem 

iii)  stochastic  Control  of  Linear  Systems  by  the  Multiple  Time  Scales  th< 

1,6.1.  Control  of  Fi  St  te  Finite-M  lory  Stochastic  Svst 

Professor  N.R.  Sandell,  Jr.  has  worked  <’>  the  finite-state  finite-  n r..o 

stochastic  ntrol  problem.  Th<  detail.1  of  thi  w<  rk  may  U found  in  i 

111  and  [:>]. 
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For  this  problem,  the  usual  tecluiigue  of  stochastic  cl  • ; ■ . >i-.  1 <•  program-  > • • : 
did  not  apply.  Instead,  opt  j i i.ility  cor..’  i tions  have  been  obt  . i • 1 by  dorr 
an  equivalent  d.  t e < t e ni  i.l ic:  optimal  contra  1 problem. 

A J-’S minimum  « Lnciple  he.  ben.n  obl;iriod  via  the  equivalent  detor- 
ministic  pr<  lem.  The  minimum  principle  has  si  jg  st.ed  th<  development  of  a 
numerical  optimisation  algorithm,  the  min-i:  algorithm  The  relation:  hip  be- 
tween the  nut  '•  iciency  of  the  minimum  principle  (which  has  been , in  general, 
only  a necessary  condition)  and  the  infori  •«> Lionel  pr;  ; -rties  of  the  pccblcr. 
has  been  invest. i ci;, ted. 

Dynamic  programing  functional  equations  for  the  FSFM  pro'  lorn  have  also 
been  obtained  from  the  equivalent  deterministic  problem.  Both  tl;c  finite 
and  infinite  horizon  cases  have  been  considered.  Du:  eri cal  solution  of  the 
functional  equations  has  been  discussed. 

To  illustrate  the  general  theory,  a problem  of  hypothesis  testing  with. 

1-bit  memory  has  been  investigated.  The  discussion  has  illustrated  the  a;  si i ca- 
tion of  control  theoretic  techniques  to  .information  processing  problems . 

This  work  has  suggested  that  optimal  decentralized  controller.*  at  ten;  •*, 
to  signal , or  communicate  with,  each  other.  Since  these  signalling  strategies 
introduce  non linearities  into  even  linca '•  control  systems,  it  is  impossible  at 
present  to  compute  the  optimal  decentralized  strategies . However,  the  nature 
of  the  optimal  solutio  n suq  . the  f i nubility  cf  designing  suboptimal  linear 
controllers  that  are  not  cor.;  1<  fely  i ■ < 1 . . i ■ d,  but  cr.  .uni  cate  across  mutual 
interfaces . 

Preliminary  investigation  indicate;  that  the  suboptimal  controller  dorian 
for  find  inter/.,'  alrret.iuv  . . . i 1 • for*v»lat'*d  ; a>-ar.-'_-r  optimize  cion 

• > ■ • ! r.  ■ i ..  • of  .•  're  is  underway. 


ju'obli 


1).  veb  . 


1 ! i pi  aimed  1 o 


test  the  resulting  I < ■ > i ; t ■ on  a nuniber  oi  • ■ 1 Is,  ' nc ] ■ i J : n a ) '■  1 oi 
the  i '-8  aircraft  at  ,>  high  anc  le  of  attack  wh  • - i • ■ ling  5 tween  th 
longitudinal  and  lat  eral  dynamic:.  becc  - .*•  • a.i  i icant . 


Rt  for-.  nni.r. 


I . N.  R.  8; ind< 'll , Jr. , Control  Finite-  . • , Finite-!1  • >ry  St< scl  ti< 

Ph.D.  Thesis,  M.I.T. , 1974. 


r:.R.  S.vtdell,  Jr.  an  i M.  Ath 
Principle" » Proc<  jin 
Houston,  Texas , j . 637-< 
of  Optimiaatioi  ’ and 


nr. . "A  ri nite-S:  U , Finite-!'.-  *y  !:inir;uv 

rt’EE  Co.ufi  i i ] ' ■ 1 Con t r I 

j a ( also  t .. 

:•!.  ic.sLio-n  ) . 


1.6.2.  One- Gt.c;  • )>-i  ly  "y  , irr? . la:-: 

In  the  classical  stochastic  control  case  (single  controllej  with  access 
to  all  past  system  me;  urement;  ) the  linear-quadratic- Gauss  Lan  (LQC)  prcbl 
provide:,  a convenient  f ran  .-work  for  a wide  class  of  practical  probl  : . In 
the  non-  classical  case,  the  solution  to  the  LOG  prcbl  ■ is  uni:  no-.  althov<f 
it  is  known  the  optim  1 control  Laws  are  in  general  nonl Ln<  a (Wit  enl  t - :. 
counter  example) . Research  has  been  directed  towards  finding  class  . oi 
problems  for  which  the  optimal  control  laws  arc  linear,  and  at  understand! r.c 
cases  in  which  the  control  lawn  are  nonlinear.  Result;  have  been  obtained  fc-i 
the  one-step  delay  I.QG  problem  and  the  control-sharing  LOG  problem.  Vhe  so’  u- 
tion  to  the  former  is  linear,  to  the  latter,  nonlinear.  The  nonlineai  tuti 

clearly  indicates  that  coding  (a  nonlinear  operation)  is  required  for  a.  ’’  • 

tion  between  the  controllers. 


Reference 
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N.R.  Sandell  and  M. 


Deci  s;  ioi!  Problems"  , 
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rat.od  1 rales.  Sj  . ally,  oi  can 


1 Ult  to  the  ' the 


!'hu'  -i  : ■ -rt  r riod  mode;  in  the-  longitudinal  dvn.n .. 


tin  i ’par  • i i •:  t>  rigid  bo'.  - and  flexur.- 


of  aircraft,  sj  ecra 


’ ' ■ /ehioli  ; nd  the  sej  i t n < < • hulei  nd  earth-i  t<  fre- 
quence irj  di  I n ivig.  tion  sy.-.?  . };  feet-  of  thin  tvpe  . re  often 

exploited  in  otsv  > ring  r.-ienc  s,  but  have  only  recently  ! , n intro- 

duced into  control  : a cn  by  l.qov  c an  .3  ..  few  otl/er  wer'-. ers 

Mr-  !l-  Tenehetzis  i nd  Prof<  ■ r N.R.  Sandell , Jr.  haw  in w tigated  t 
applic  at.  eon  of.  multijd.c  t*.ir:o  scale  methods  t.o  tie  stoc  'mst  j.c  linear  v,  c.u- 


ator  problei  . Us  in  j th  method  logy  of  singular  j rturbai  on  ti 


■ , a 


hierarchically  structured  controller  Las  been  obtained  the4.  is  c-;  • i-  •» j jn 


the  limit  as  the  separation  of  the  time  series  become  infinite. 


ue  higne  r 


levels  of  the  controller  deal  with  the  slower  system  dynamics  and  d.:>  not 
require  any  information  from  the  lower  levels.  The  lower  levels  of  the 
controller  deal  with  the  faster  system  dynamics  and  require  information  f.-a  . 
higher  lew  Is.  Thus  the  higher  level  portion  of  the  control  algorithm  can 
bo  iterated  at  a much  slower  rate  than  would  be  possible  by  a single  time 
scale  dec  -a  with  an  attendant  saving  in  on-line  comput  it ion.  Signif  . . • ■ : i • . 
this  sav ' ng:.  is  not  achieved  at  the  expense  of  additional  off-line  comput  at  ’ 
Indeed,  the  off-line  computations  of  the  multiple  time- scale  dcni«;r  are  redv. 
in  size  and  are  better  conditioned  than  those  of  the  single  time  scale  do.  i< 
Tlte  results  have  been  documented  in  the  Ji.S.  thesis  of  Kv.  Tern-hotels.. 

Re  f a re n re 

1 ■ ‘ 1 ■ ' • ■ • : < Perturbation  Methods  > I rali  ' . . ■ , . . " f 

’ ; , ‘ ' Jl  n<  ; Thesis,  Massachi  I Teel 

nolo  , C il  icin  etts) . 
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.1.7.  i v- ! corti  oj 

The  n<  ; for  adaptive  control  Ls  nei  ssitated  by  the  ne  1 t ' : into 
account  explicit  vari  ntioms  in  the  parameters  of  the  cm.  loop  ay.  . Sov  ■ 
ad.ip  tive  technique:.  are  available  in  the  literature  c -.eh  j • rnirir.v  d i : fc  • er.t 
amounts  of  real  ti vie  conputation. 

Wnat  if.  not.  clear  from  an  engineering  vj cvpoint  ' : vj ae  -rfive  c ’ttr  1 
should  be  u • d.  If  on  follows  the  modern  trend  in  evaluating  the  perron  . 
oi  an  tdaptive  stochastic  control  algorithm  not  only  by  it!  Ldentifi  iti<  n 
accuracy , but  rathe*  by  the  performance  improvement  (as  . . sured  by  a wel 1 
defined  perf<  in  oi  the  overall  system,  then  it  is  desirable  to  h v< 

an  idea  of 


(a)  the  robustn  ss  of  a non- adaptive , i.e.  fixed  g di  , sti  ' ..  ..  conti 

algorithm 


O')  a "worst-ca  e"  type  of  performance  that  migh  be  expected  if  ] 
identification  cannot  be  carried  out. 

To  properly  answer  these  questions  is  the  subject  of  a long  rang-,  urn  reh 
program.  Several  resear  1 proji  ts  have  been  undertaken  to  answi  i those 
tions.  Wo  feel  that  partial  res  ull  s v.v  have  obtained  pr<  vide  i ) for  iu  lei 

standing  not  only  the  issue  of  when  one  .should  use  adaptive  control , but  hope- 
fully shed  sonv.  light  upon  the  dove  lopm.nt  of  adaptive  control  al>  •••rifhnr.s 
which  are  computationally  more  efficient  than  present  ili.il  i.v-thod;  . 

Of  fundamental  importance  i ; the  interaction  of  ay:  ten  unc  rtainty,  as  ex- 
hibited! Large  parat  ter  variations  in  th  open  3 r.amics,  and  th<  tj 


of  J para'  .'t’lo  that  appear 


iai.ee  index  to  be  optimised. 


in  l-.h.  ■ per  fori- 


Kobuslm  of  1J  n-  . r t_j  c I ’ i > ■ ns 

In  section  2.5,  Part  3 o'.  I!  report,  wo  hav'  i.;.’.;cri.K-3  result-:  on 
robust  no;;:'  oi  by  donigms.  The  implication!;  of  thc-s-  renul  t:  on  ade;  t ) vc 
control  have  l-eon  invent i gated. 

Pi  obi ng  <n  ' hoarni  q in  Adaptive  Control 

See  section  2 . , Part  3 of  the  report . 

Ada; tivi  and  Co  uni cation;  At;  t : ( I ! ' i ■ - 3 < si cal  Control  Th 

Mr.  p ,vi d Cas tagnon  with  Professors  Ath.  ns  and  famic-.ll  have  Ir  on  invisti- 


gating- 

adca  ti  va  and  comi. 

tun  Lcat  Lc  i asp<  - ts 

of  nos-  cl - 

sic  1 stoclia. 

-.tic  contr 

theory. 

These  problems 

arc  o).s>  ,'Otcri:.( •<'. 

by  infoio.nl 

ion  ; -attorns. 

avai ' < 1 s 

to  the 

controller  wb.i  ch 

are  not  neces s a r i 

Iv  of  the  ty 

; s occurring 

.in  stoch  - 

control. 

Tlia  problem  considered  vs  - on-  when-  perfect  rot  a i 1 is.  not.  pro:  ent  . 'Jui-' 
is  a stochastic  control  problem  whereby,  through  memory  limitation!--  or  oth--r 
factors,  the  controller  may  3 oae  inform U. ion  from  t.i-  e to  time.  A mathc-rub  1 o -.3 
framework , similar  to  witsenhausen' ; was  dovelo;  ••  - d , using  this  fr.  :woA  , 
general  I3c.-cor.sary  Conditions  were  developed  for  the  cri.-ti-nco  of  a solut.ii  a. 
Similarly,  several  results  based  on  duality  properties  wen  esl  ' Lish  d Wni 
produced  some  feasibl*  techniqu*  - to  obt.  in  i ilutions  i<-  the  prol  1 m. 

Mu 3 1 i pi  Model  A.‘  '!'  1 ivc_CanJ  • <>1 

Professor  M.  Athans,  Dr.  K--P  Dunn,  in  i Mi.  B.F.  Gong  hav  studied  cevi  - 
theoretical  issues  related  to  adaptive  control  of  nonlinear  stood.  : tic  y - - 
with  - i.ii  emphasis,  on  probl  e:  .r  assoc.1  .ted  \ i th  tin  - - ■ - i < , n of  ad.ipt.ivc 
stability  augmentation  systems  foi  aircraft.  uation  > tin  F-8  Di  il 

Fly-P.y-Wi  re  aircraft  have  1-  on  us,  3 as  tin  u • vt-'.-i  el«  for  usd  - tandin-s  l ' 

1 as  • c ■ ■■  U<  s. . 


J 


Particular  attentia  1 ■ < » to  th  called!  tipl<  ] 

tivc  Control  (MMAC)  m whit  • » is  testing  ideas  with  Lin  i 

Quadratic  Gaussian  de  i t • . 1 .■ . i u<  to  < > : 1 a nonliut  • : • ■ ’ ' ■ vi  1 ■ ; 11 

system.  1 i 1 1 . ■ i till  [uirci  th  th  lineal  z<  > I < ■ mstic  - c ■ 

1 he  nonl inoai  system  (-...,  th(  lit  ■ ■ z<  < jua  a ■ oi  th<  aircr*  t)  are 
availa)  Le  at  several  op  rating  < n Lon  (in  the  :ast  c aircra  t . res 

pond  Lng  to  equil  briuui  i ht  at  different  altitude  and  ac  1 1 1 >ers) . 

There  are  two  basic  t tradeoffs  that  have  to  be  consider*  d. 

1 ) From  th<  point  oi  v -w  oi  control  systt  1 < uu  , t ! ■ • 1 - 1 v 1 ■ 1 ■ 

number  oi  lino«r  models  . . : : ble,  the  1 itt  r 1 control  system. 

2)  From  the  point  of  vi*  w of  cov.tr-;  . ! b-n,  t!i<  greater  th.  nr.  r o!  1 . ^ 1 , 

the  larger  the 


(a) 

CCl 

e 1.;.  iaor>  i*  guir.  m-  nts 

(b) 

re. 

il-tim a co; .ul:.-itions . 

What 

Wt! 

lt.r/o  been  developing  is  a 

sys 

tematic  approa  ' ■ , . 

;r -toying  ( 

'y  ramie 

programming , to  arrive  at  a scientific  way  oi  selecting  the  numb<  o ! •• 

th<  MMAC  apj  - - 'h  taking  into  accoui it  the  t • • L<  • '■  1 > ‘ perform  11  c<  vs.  * 

ti.oiial  complexity.  The  results  of  this  work  hav  been  docum  anted  in  the  R.M. 
thesis  of  IS.)'.  Gong. 

Reft  nee 

l.  b.F.  Gong,  "On  Adaptive  Stocha  ti<  Control",  H.S.  T - , May  197<  , 

Massachusetts  Ins tituti  of  T<  In  logy , C ige , Massachusetts. 

1 . j . . Infiii-i  Di  ei  Lo  ] eai 

Over  th"  past  fiv-  y.  ,.r;  Pro!  ■ nor  M.itlci  k-..  v rl.e.1  on  ) 'rob! cm:  • f • 

I- ton  and  control  for  a large  el. a;  : of  linear  infinite  diroem.ion.il  sys  tem  . 
cl,  . in  1 - - . . 1 ■ ■ t ; 1 ay , s’  tom  ■ ■ 1 ■'  ! > 


and  systi sms  described  by  second  or<  i : j rl  Li<  equal  on.  A the  ry  which  i 
formally  analogous  to  the  linear-qu  . • : Gau  j i theory  fo  finite  di 

sional  linea r sys tew  is  n av«  : 1 . 1 • 1 < . ■ t<  i Ls  of  this  c m ! e fc  ...  in  tl 

foil  h coin  in  q wonoqrai !i  [1]. 

Re fcri  ’icc 

1.  A.  Bensoussan,  M.C.  D lfour , and  Hitter f RcpreseiH  i < > md  C ■ ro'J 

of  Lineal  Infinite  Dimensional  Systei  , to  appear. 
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